Skip to content

fix(rust): unblock four adaptive-sampling parametric tests#7007

Draft
bm1549 wants to merge 1 commit into
mainfrom
brian.marks/rust-adaptive-sampling-unblock
Draft

fix(rust): unblock four adaptive-sampling parametric tests#7007
bm1549 wants to merge 1 commit into
mainfrom
brian.marks/rust-adaptive-sampling-unblock

Conversation

@bm1549
Copy link
Copy Markdown
Contributor

@bm1549 bm1549 commented May 22, 2026

Motivation

Enables four parametric tests for Rust now that DataDog/dd-trace-rs#227 fixes the underlying precedence and adds DD_TRACE_SAMPLE_RATE support.

What does this PR do?

Removes four missing_feature entries from manifests/rust.yml:

  • TestDynamicConfigSamplingRules::test_trace_sampling_rules_override_rate
  • TestDynamicConfigSamplingRules::test_trace_sampling_rules_with_tags
  • TestDynamicConfigV1::test_trace_sampling_rate_override_env
  • TestDynamicConfigV1::test_trace_sampling_rate_with_sampling_rules

Do not merge before DataDog/dd-trace-rs#227

The parametric scenario runs against both the dd-trace-rs main build (rust dev) and the released crates.io version (rust prod). Both fail on these four tests until dd-trace-rs#227 lands.

Sequence:

  1. Merge dd-trace-rs#227.
  2. Mark this PR ready for review; re-trigger CI. rust dev goes green.
  3. After the next dd-trace-rs release contains the fixes, rust prod also goes green.

Test plan

  • CI confirms the four tests pass against the dd-trace-rs main build after Fix ARM node #227 merges.
  • No other rust manifest entries regress.

Enables tests that now pass with the adaptive-sampling precedence fix
and DD_TRACE_SAMPLE_RATE support shipped in dd-trace-rs PR #227:

- TestDynamicConfigSamplingRules::test_trace_sampling_rules_override_rate
- TestDynamicConfigSamplingRules::test_trace_sampling_rules_with_tags
- TestDynamicConfigV1::test_trace_sampling_rate_override_env
- TestDynamicConfigV1::test_trace_sampling_rate_with_sampling_rules

The Rust tracer now:
- Respects the documented multi-source precedence (env rules survive
  an RC-rate-only update; the synthetic catch-all built from
  tracing_sampling_rate uses default provenance, DM -3).
- Parses DD_TRACE_SAMPLE_RATE as a global catch-all rate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@bm1549 bm1549 added the ai-generated The pull request includes a significant amount of AI-generated code label May 22, 2026
@github-actions
Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

manifests/rust.yml                                                      @DataDog/apm-rust

@datadog-datadog-prod-us1-2
Copy link
Copy Markdown

datadog-datadog-prod-us1-2 Bot commented May 22, 2026

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 3 Pipeline jobs failed

Testing the test | System Tests (rust, dev) / parametric / parametric (1)   View in Datadog   GitHub Actions

🔧 Fix in code (Fix with Cursor). 5 failed tests due to assertion errors on the sampling rate.

🧪 5 Tests failed

tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules.test_trace_sampling_rules_override_rate[library_env0, parametric-rust] from system_tests_suite   View in Datadog (Fix with Cursor)
assert 1.0 == 0.9 ± 9.0e-07
  comparison failed
  Obtained: 1.0
  Expected: 0.9 ± 9.0e-07

self = &lt;tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules object at 0x7f899e368b60&gt;
test_agent = &lt;utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f89699b60f0&gt;
test_library = &lt;utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f8969b26de0&gt;

    @parametrize(&#34;library_env&#34;, [{**DEFAULT_ENVVARS}])
...
tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules.test_trace_sampling_rules_with_tags[library_env0, parametric-rust] from system_tests_suite   View in Datadog (Fix with Cursor)
assert 0.55 == 0.8 ± 8.0e-07
  comparison failed
  Obtained: 0.55
  Expected: 0.8 ± 8.0e-07

self = &lt;tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules object at 0x7f899e368e60&gt;
test_agent = &lt;utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f8969820860&gt;
test_library = &lt;utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f8969822150&gt;

    @parametrize(
...
View all 5 test failures

Testing the test | System Tests (rust, prod) / parametric / parametric (1)   View in Datadog   GitHub Actions

🔧 Fix in code (Fix with Cursor). 5 failed tests due to assertion error in sampling rates. Obtained: 1.0, Expected: 0.9 ± 9.0e-07 at tests/parametric/test_dynamic_configuration.py:234.

🧪 5 Tests failed

tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules.test_trace_sampling_rules_override_rate[library_env0, parametric-rust] from system_tests_suite   View in Datadog (Fix with Cursor)
assert 1.0 == 0.9 ± 9.0e-07
  comparison failed
  Obtained: 1.0
  Expected: 0.9 ± 9.0e-07

self = &lt;tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules object at 0x7fd39aea0860&gt;
test_agent = &lt;utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7fd3661159a0&gt;
test_library = &lt;utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7fd366115010&gt;

    @parametrize(&#34;library_env&#34;, [{**DEFAULT_ENVVARS}])
...
tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules.test_trace_sampling_rules_with_tags[library_env0, parametric-rust] from system_tests_suite   View in Datadog (Fix with Cursor)
assert 0.55 == 0.8 ± 8.0e-07
  comparison failed
  Obtained: 0.55
  Expected: 0.8 ± 8.0e-07

self = &lt;tests.parametric.test_dynamic_configuration.TestDynamicConfigSamplingRules object at 0x7fd39aea0b30&gt;
test_agent = &lt;utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7fd36618db50&gt;
test_library = &lt;utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7fd366081580&gt;

    @parametrize(
...
View all 5 test failures

Testing the test | all-jobs-are-green   View in Datadog   GitHub Actions

🔧 Fix in code (Fix with Cursor). Some CI checks or statuses failed for System Tests (rust, prod) and (rust, dev) check runs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a8dc09f | Docs | Datadog PR Page | Give us feedback!

gh-worker-dd-mergequeue-cf854d Bot pushed a commit to DataDog/dd-trace-rs that referenced this pull request Jun 1, 2026
)

# What does this PR do?

Fixes adaptive-sampling Remote Config in `datadog-opentelemetry`, rebuilt on top of PR #154. Also adds `DD_TRACE_SAMPLE_RATE` support.

Before this PR, four things were broken:

1. **`tracing_sampling_rate` from RC was ignored.** The handler only acted on `tracing_sampling_rules`; a rate-only payload installed nothing.
2. **RC list-shape `tags` were rejected.** RC encodes `tags` as `[{"key": "env", "value_glob": "prod"}]`, but `libdd_sampling::SamplingRuleConfig::tags` only accepted the map shape, so the parse errored and the whole update was dropped.
3. **Env `DD_TRACE_SAMPLING_RULES` were wiped on every RC update.** `update_sampling_rules_from_remote` does a full override, so any RC change replaced env rules, even when RC only sent a global rate.
4. **`DD_TRACE_SAMPLE_RATE` had no effect.** No binding existed.

# Motivation

End-to-end adaptive sampling didn't work.

# What changed

**Composition.** `ApmTracingHandler::process_config` now follows the multi-source precedence model:

| env rules | env `DD_TRACE_SAMPLE_RATE` | RC `tracing_sampling_rules` | RC `tracing_sampling_rate` | Effective rule chain |
|---|---|---|---|---|
| present | any | absent / null | absent / null | `env_rules` |
| present | unset | absent / null | present | `env_rules + catch_all(rc_rate)` |
| present | set | absent / null | absent / null | `env_rules + catch_all(env_rate)` |
| present | set | absent / null | present | `env_rules + catch_all(rc_rate)` |
| any | any | non-empty array | absent / null | `rc_rules + catch_all(env_rate)` if env_rate set |
| any | any | non-empty array | present | `rc_rules + catch_all(rc_rate)` |

The synthetic catch-all uses libdatadog's default provenance, mapping to DM `-3` (LOCAL_USER). See the "legacy behavior" comment in [test_trace_sampling_rules_override_rate](https://github.com/DataDog/system-tests/blob/main/tests/parametric/test_dynamic_configuration.py#L872).

**`DD_TRACE_SAMPLE_RATE`.** New `Config::trace_sample_rate(): Option<f64>`. When set, the sampler installs an implicit catch-all so `DD_TRACE_RATE_LIMIT` applies. Unset means no catch-all (libdatadog's no-rule path samples at 100%).

**Tag normalization.** RC encodes `tags` as the list shape `[{"key", "value_glob"}]`. This is parsed natively by `libdd-sampling` ≥ 2.1.0 (DataDog/libdatadog#2033), so this PR bumps `libdd-sampling` 1.0.0 → 2.1.0 (pulling `libdd-common` → 4.2.0) and no in-tracer normalization is needed. An earlier revision carried a `normalize_rc_tags` shim for this; it has been removed now that the upstream release is available. Regression coverage: `test_handler_rc_rules_with_list_tags_applied` (list-shape tags apply, tags preserved as a map) and `test_handler_malformed_tags_rejects_update` (malformed list entries still rejected wholesale, not silently broadened).

**Fail-closed behavior.** When libdatadog rejects an update (malformed tags, out-of-range rate), `process_config` returns `Err` so the RC dispatcher reports `apply_state=3` and the prior policy survives. Out-of-range RC `tracing_sampling_rate` (outside `[0.0, 1.0]`) and non-numeric values are rejected up-front.

**Env/code rate validation.** `DD_TRACE_SAMPLE_RATE` (and the programmatic `set_trace_sample_rate`) get the same range check: only finite values in `[0.0, 1.0]` are honored; out-of-range values are logged and treated as unset rather than installed as a catch-all rule that libdatadog would clamp (negative ⇒ drop all, > 1.0 ⇒ keep all).

**Target check.** A config's `service_target` is honored before applying: a payload whose specific (non-`*`) `service`/`env` doesn't match this tracer — primary service or an advertised extra service, compared case-insensitively — is ignored, so a mistargeted RC delivery can never change this service's sampling. Mirrors dd-trace-py/go.

# Additional Notes

- Four parametric tests unblocked by this PR. Companion PR DataDog/system-tests#7007.
- Coordinated with @iunanua's PR #222 (libdatadog RC client wiring). #227 lands first; #222 rebases on top.



Co-authored-by: brian.marks <brian.marks@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-generated The pull request includes a significant amount of AI-generated code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant